Search for: All records

This paper considers an infinite-horizon Markov decision process (MDP) that allows for general nonexponential discount functions in both discrete and continuous time. Because of the inherent time inconsistency, we look for a randomized equilibrium policy (i.e., relaxed equilibrium) in an intrapersonal game between an agent’s current and future selves. When we modify the MDP by entropy regularization, a relaxed equilibrium is shown to exist by a nontrivial entropy estimate. As the degree of regularization diminishes, the entropy-regularized MDPs approximate the original MDP, which gives the general existence of a relaxed equilibrium in the limit by weak convergence arguments. As opposed to prior studies that consider only deterministic policies, our existence of an equilibrium does not require any convexity (or concavity) of the controlled transition probabilities and reward function. Interestingly, this benefit of considering randomized policies is unique to the time-inconsistent case. 

A new definition of continuous-time equilibrium controls is introduced. As opposed to the standard definition, which involves a derivative-type operation, the new definition parallels how a discrete-time equilibrium is defined and allows for unambiguous economic interpretation. The terms “strong equilibria” and “weak equilibria” are coined for controls under the new and standard definitions, respectively. When the state process is a time-homogeneous continuous-time Markov chain, a careful asymptotic analysis gives complete characterizations of weak and strong equilibria. Thanks to the Kakutani–Fan fixed-point theorem, the general existence of weak and strong equilibria is also established under an additional compactness assumption. Our theoretic results are applied to a two-state model under nonexponential discounting. In particular, we demonstrate explicitly that there can be incentive to deviate from a weak equilibrium, which justifies the need for strong equilibria. Our analysis also provides new results for the existence and characterization of discrete-time equilibria under infinite horizon. 

Abstract An unconventional approach for optimal stopping under model ambiguity is introduced. Besides ambiguity itself, we take into account howambiguity‐aversean agent is. This inclusion of ambiguity attitude, via an‐maxmin nonlinear expectation, renders the stopping problem time‐inconsistent. We look for subgame perfect equilibrium stopping policies, formulated as fixed points of an operator. For a one‐dimensional diffusion with drift and volatility uncertainty, we show that any initial stopping policy will converge to an equilibrium through a fixed‐point iteration. This allows us to capture much more diverse behavior, depending on an agent's ambiguity attitude, beyond the standard worst‐case (or best‐case) analysis. In a concrete example of real options valuation under model ambiguity, all equilibrium stopping policies, as well as thebestone among them, are fully characterized under appropriate conditions. It demonstrates explicitly the effect of ambiguity attitude on decision making: the more ambiguity‐averse, the more eager to stop—so as to withdraw from the uncertain environment. The main result hinges on a delicate analysis of continuous sample paths in the canonical space and the capacity theory. To resolve measurability issues, a generalized measurable projection theorem, new to the literature, is also established.